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The undergraduate liberal arts program is assumed to promote sophistication in thinking. At the same time, certain 
levels of cognitive ability are assumed to be present in students accepted into selective undergraduate colleges. 
Beginning in the fall of 1991, a long-term research study was undertaken at a small residential liberal arts college 
that was designed to test both of these assumptions. Specifically, students were assessed for formal operational 
ability and intellectual flexibility at the beginning of their freshman year, as sophomores, and finally, -as seniors. 
Although it is not possible to differentiate them from those changes that might have occurred just because of the 
passage of time, these findings do suggest that the young adults in this sample did become less concrete and more 
formal thinkers over their college careers (percent concrete thinkers decreased from 21.8% to 7%). Participants also 
became less absolute and more evaluative in their approach to problems. 



The undergraduate liberal arts program is assumed to promote sophistication in thinking. At the same time, certain 
levels of cognitive ability are assumed to be present in students accepted into undergraduate colleges such as 
Allegheny. Beginning in the fall of 1991, a long-term research study was undertaken that was designed to test both 
of these assumptions. Specifically, students were assessed for formal operational ability and intellectual flexibility 
at the beginning of their freshman year, as sophomores, and finally, as seniors. Eventually, three cohorts of students 
(those matriculating in 1991, 1992, and 1993) were included in the study. 

This effort was designed to provide information about the efficacy of our curriculum as well as to meet the 
evaluation requirements of several grants and outside agencies such as Middle States. 

It is important to remember that formal operational ability and intellectual flexibility are but two of the many skills 
and abilities which our curriculum is designed to foster. For example, the proposed testing did not directly examine 
either writing or creative abilities, among others. This effort should be seen as an attempt to begin the process of 
evaluating our curriculum by looking at some global achievements and not as an attempt to examine every goal of 
our teaching. 

There is some disagreement in the literature on the appropriateness of the model proposed by Jean Piaget to adult 
intelligence. Some researchers and theorists. Perry in particular, suggest that college students should be seen as 
being in a "postformal” period rather than in the formal operations stage. Consequently, a paper-and -pencil measure 
of adult intellectual development that focused on absolutism, relativism, and evaluativism was added to the test 
battery in the fall of 1992. 
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Deflning Formal Operations and Intellectual Flexibility 

Formal Operational Abilities 

Formal operations is the highest stage of cognitive development in Piaget’s (1950) model, entailing the ability to use 
abstractions in the construction of hypothetical perspectives. The stage immediately prior to formal operations. 
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concrete operations, entails a dependency on concrete experience. Undergraduate studies typically depend heavily 
upon cognitive abstractions. However, current research suggests that as many as 40% of college freshman either 
lack formal operational ability or that they are in a state of transition unable to make effective use of their newly 
acquired abilities. Measurement over time was designed to determine if formal operational ability is acquired and/or 
more fully developed during the course of our undergraduate program. 

Intellectual Flexibility 

Under many different names, disciplined flexibility in analytical thinking is usually thought of as a hallmark of the 
liberally educated person. Sometimes it is called relativism of judgement or commitment to one position while 
understanding diverse views (Perry, 1970). Sometimes it is simply called reasoning ability. In any case, it involves 
the ability to see the elements of truth in all sides of a controversy, to analyze arguments, and to construct coherent 
ways of evaluating those arguments. 

Adult Intellectual Development 

A number of investigators, including Perry, have studied the development of postformal operational thought. Perry 
showed that students began college with a dualistic belief in absolute right and wrong and then enter a period in 
which they accepted multiple possibilities while contradictorily maintaining a belief in ultimate truth. Next, these 
students advance through a phase of relativism in which they believe there was no absolute right or vvrong, and that 
all knowledge was contextually based. Subsequent to their period of relativism, the students end their collegiate 
tenure with a commitment to knowledge based on the best personal judgement possible. 

Kitchener and King (1981) developed their Reflective Judgement Interview (RJI) from the Perry tradition 
specifically to provide a more standardized, a more reliable, and a more exclusively cognitive measure of these 
levels and transitions. Seven intellectual positions in the RJ model can be described in terms of three broad levels of 
Absolutism, Relativism, and Evaluativism. 

Individuals within the Absolutism level believe in the existence of absolute knowledge and in the direct availability 
of that knowledge through appropriate sources. Absolutists consider appropriate sources to be authorities and their 
own sensory and mental experiences. 

For the Relativist, knowledge has become contextual, and tends to loose its truth-value. Only in concrete, scientific 
areas of knowledge are the idea of truth even slightly meaningful and often even that is doubted. 

At the Evaluativism level, individuals began to critically evaluate ideas, evidence, authorities, and arguments. For 
the Evaluativist, knowledge is explicitly justified by the method through which it was obtained. 



Measuring Formal Operations and Intellectual Flexibility 

Measuring Formal Operations 

Formal operations was be assessed by the Arlin Test of Formal Reasoning (ATFR, Arlin, 1984). This 32 item, 
multiple choice test takes about 45 minutes but is not timed and students are permitted finish all the items. The 
items in the test were designed to closely parallel the description of tasks employed by Inhelder and Piaget (1958) to 
clinically assess an individual's ability to use the eight specific concepts associated with the stage of formal 
operations. Although the names used for these eight formal concepts are reminiscent of the science and 
mathematics experiments which were first used to illustrate Piaget's formal logic system, it is important to realize 
that they also represent thinking processes used in all intellectual endeavors. Each concept represents a form of 
thinking and not necessarily the content of that thinking. They are known by the fact of their varied applications 
rather than being concepts in the narrow sense of the term (Inhelder & Piaget, 1958, p. 309). This instrument was 
chosen over several others because of its ease of administration and because of the wealth of published research that 
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can be used to provide comparative data for this effort. 

The eight formal schemes are described below using details provided in the ATFR manual. 

1) Multiplicative Compensations is the concept which supports the understanding that when there are two 
or more dimensions to be considered in a problem, gains or losses in one dimension are made up for by gains or 
losses in the other dimensions. A common question that requires this kind of thinking for its answer is: "If I make 
this change what effect does this have on ...?” 

2) Correlations is a concept that implies the ability of a student to conclude that there is or is not a causal 
relationship, whether negative or positive, and to explain the minority of cases by inference of chance variables. 
Correlational reasoning provides answers to questions that begin: "What is the relationship ...?" 

3) Probability is a concept that supports the ability to develop a relationship between the confirming and 
the possible cases. Students are using this concept whenever they ask themselves the question "What is the 
possibility that...?" or "What's the likelihood that ...?" 

4) Combination reasoning involves the concept of generating all possible combinations of a given number 
of variables, choices, events, and scenarios when a problem solution requires that all possibilities are accounted for. 
Students use this concept when they ask the question: "What are all possible combinations of..?" 

5) Proportional reasoning is the fifth formal scheme. It is defined as a mathematical concept that 
involves the ability to discover the equality of two ratios that form a proportion. Aristotle in his poetics defined an 
analogy as a statement where the second element is compared to the first, as the fourth is the third. Is this not a 
restatement of the definition of proportionality? Interpreting analogies and complex poetic examples require 
multiple classifications and their coordination. Studies (Billow, 1975; Arlin, 1977, 1978) show that the proportional 
reasoning concept appears to be a necessary but not sufficient condition for interpreting analogies and expanded 
metaphors beyond their more literal level. 

6) Forms of Conservation Beyond Direct Verification and the seventh scheme. Mechanical Equilibrium 
would appear to be narrowly defined in terms of scientific concepts but they too have implications for other areas of 
inquiry. For example, consider the saying: "By their deeds you will know them." This dictum suggests a type of 
inference is required to get at the essence of personal definition that one cannot directly see. In a sense it requires 
the use of the scheme of forms of conservation beyond direct verification in its broadest sense. 

7) Mechanical Equilibrium requires the ability to simultaneously make the distinction and the 
coordination of two complementary forms of reversibility-reciprocity and inversion. Most economic theories 
presuppose these types of trade-offs to maintain equilibrium in the system. The writing of novels and the 
interpretation of complex plots in plays, all represent this delicate balance. 

8) The Coordination of Two or More Systems or Frames of Reference is one of the most complex 
schemes, and yet is applications are wide-ranging. It may well be the scheme which acts are a bridging concept 
between formal and any type of post-formal thinking. 

Scoring the ATFR. Based on the test score, students are assigned to one of five cognitive levels as defined below. 

1) Concrete represents performance on the formal tasks which is best described as providing no evidence 
of abstract reasoning and some difficulty with reasoning skills that are problem specific. 

2) High Concrete represents performance that provides some evidence of a systematic approach to 
problems but not evidence of forming a general rule or abstraction from the problems. 

3) Transitional students provide some evidence of a systematic approach to the problems and some use of 
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abstractions and inferences but the performance is quite inconsistent. 

4) Low Formal students give clear evidence of three-to-five of the formal schemes being present in their 

thinking. 



5) High Formal students give clear evidence of most of the formal schemes being in evidence in their 

thinking. 

Measuring Intellectual Flexibility 

Intellectual flexibility will be measured by using a variation of a procedure proposed by Winter, McClelland, and 
Stewart in their book A New Case for the Liberal Arts (1981). Their measure, called the Analysis of Argument Test, 
duplicates as much as possible the familiar situation in which a complex, potentially emotional issues is being 
discussed. 

Students are first given a brief quotation that expresses a strong, extreme position on a controversial and emotional 
issue. The issue chosen is one that was expected to produce a strong negative reaction in most college students. 
After reading the quotation (300 to 400 words), they are given five minutes to write a response: "Assume that you 
argue against it. Your response can be of whatever form and nature that you think is most satisfactory and 
appropriate." After writing their responses, without any prior mention, subjects are then told to turn the page, where 
the next instruction reads as follows: "Write a defense of the quotation-including if you wish, argument against 
your criticism that you have Just written. Again, your defending response can be of whatever form and nature that 
you think is most satisfactory and appropriate." 

The same article was used as a stimulus to this writing throughout all the years of the study. The topic of the article 
was drinking among college students and in quite strong and emotional language, it made the point that college aged 
students were much too immature to drink alcohol and that colleges must take an active and intrusive role to limit 
the consumption of alcohol. Hence, the Attack essay would try to refute this point while the Defense essay would 
support it. 

Winter, et al. (1981) have developed a scoring system that they claim can reach 90%+ agreement among trained 
scorers. 

Measuring Adult Cognitive Development 

Factor analysis of items derived from Kitchener and King’s (1981) descriptions of interviews of college age and 
older adults revealed three factors, interpreted as representing three developmentally related epistemic strategies. 
Absolutism, Relativism, and Evaluatism. Newman and Martin (1989) have made available a sixty-item paper-and- 
pencil scale {Scale of Adult Intellectual Development—SAID). The measure has proven quite reliable and has been 
shown to have convergent validity by predicted correlations between SAID and Dogmatism, Locus of Control, 
Desire for Control, Need for Cognition, and Erwin’s Scale of Intellectual Development. 

Students at the Absolutist level exhibit a simple acceptance of a viewpoint with reasoning. Beliefs that are consistent 
with, or similar to, previously held beliefs, are accepted. In the parlance of current epistemological discussion. 
Absolutists are foundationalist in the extreme. 

Among other things, the Relativist level shows a process of individual choice in a context where everyone is thought 
to be entitled to their own opinion. Instead of consistency with a single perspective, an appreciation of, or an 
accommodation to, a more complete array of inconsistent perspectives is valued at this level. 

At the Evaluativist level, decisions are based on critical evaluations of evidence, authority, and argument. 
Evaluativists move toward making Judgements in the context of the tension constituted by the Joint appreciation of 
the clashing epistemological values of consistency on the one hand, and completeness on the other. Evaluativism 
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represents a move beyond foundationalism and relativism. 

The SAID consists of 60 items, each consisting of two or three sentences. Students are asked to read each item and 
then indicate on a scale of 1 to 7 whether the item was Strongly Like Me (7) or Strongly Unlike Me (1) or some 
where in between. 

Nineteen items are included in the Absolutist subscale, 20 in the Relativist subscale, and 21 in the Evaluativist 
subscale. An example of an item from each of the scale is given below. 

Item from the Absolutist subscale: I have been surprised many times to find out how differently people can feel 
about straight-forward issues. I think that when it gets right down to it, there is really only one right view. That’s 
why most people naturally feel the same way. 



Item from the Relativist subscale: I’m willing to stand behind the decisions I have made because I look at problems 
from all sides before I make a judgement. Sometimes, standing up for my ideas results in conflicts with other 
people. I’m able to take that responsibility for sharing my views, but I don’t feel like I must change their opinions 
even if my ideas are better. 

Item from the Evaluativist subscale: It’s not possible to say that someone is totally right or wrong in what they think. 
Issues are so complex it would be easy for me to be wrong; so, how could I say someone else is? I might believe 
something more because of the reasons for it, but I would say it might be better, not right. 



Available Data 

At least some data was collected from each of 1,777 students who matriculated at Allegheny College in the fall of 
1991, 1992, and 1993. Freshmen testing was done during orientation (before the first week of class) while 
sophomores were tested on the last day of their LSW course in the fall or spring of their second year. Seniors were 
tested as part of a senior exit interview procedure. For most seniors this testing was done in the spring semester but 
for fall graduates it was done Just before they finished their final semester. 

At each testing session (freshmen, sophomore, and senior) measures of formal reasoning, intellectual flexibility, and 
intellectual development were taken. The Arlin Test of Formal Reasoning provides a raw score that allows the 
student to be assigned to one of five cognitive levels (concrete, high concrete, transitional, low formal, and high 
formal). 

The test of intellectual flexibility required that students write two essays, one defending and one attacking the 
position taken in a short, newspaper style article. The essays were read and scored by trained raters employed by 
McBer and Company in Boston. The scoring procedure, developed by Winter and McClelland, provides for scores 
ranging from -4 to +6. 

A +1 would be given, for example, when an essay focused on the logical and logical errors of the statement (attack) 
or when the writer reworked or delimited the original statement so as to defend it (defense). On the other hand, a 
score of-1 would be given if the writer simply provided a series of criticisms that were not organized around a 
central principle, insight, or focus (attack) or when, in the course of defending the article, the writer introduced new 
arguments not given in the original statement. 

The SAID provides scores on three subscales, one for each of the epistemic strategies. Absolutism, Relativism, and 
Evaluatism. 

In addition to these measures described above, demographic information was obtained on students who completed 
the surveys from the Allegheny College database. The following data where therefore available for analysis on 
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most, but not all, students: 

• SAT scores (recentered, ACT scores were converted to equivalent SAT scores where possible) 

• Gender 

• Ethnic group 

• High school rank 

• Size of high school 

• LSW200 grade 

• Cumulative GPA 

• Initial interest (field of study the student might like to pursue) 

• Major at graduation 

• Change in interest (did initial interest differ from major?) 



Data Analysis 

Given the rich data set provided by this project, there are virtually hundreds, if not thousands, of possible hypotheses 
and questions that might be examined. However, this project was initially conceived to examine the relationship, if 
any, between intellectual flexibility, initial interest, college major, and retention. However, before those more meaty 
hypotheses are examined, let's try to get a picture of how Allegheny students look on the measures taken and how 
those scores changed over time. 

Since there were no significant differences or trends from one cohort to the next, data from all three of the three 
classes were combined for the analyses provided below. 

Formal Operations 

Recall that results of the ATFR place students in one of five categories ranging from Concrete to High Formal that 
indicates their level of abstract thinking. Figures 1 and 2 present the proportion of freshmen, sophomores, and 
seniors in the various cognitive stages. Several things are worth noting. 

First, almost 25% of the freshmen in this sample were in the Concrete or High Concrete stage. Although this 
certainly presents problems to those faculty attempting to cover abstract material in their classes, this proportion is 
just about equal to that reported across the country for college freshmen. 

Second, and on a more positive note, the proportion of students in these lower stages declines significantly by the 
senior year where the proportion of High Formal operators has doubled from 14% in the freshmen year to 28%. 

Third, although there are still some students who score in the High Concrete stage as seniors, no students in our 
survey scored as Concrete thinkers as seniors. 

In summary, this data indicate that although we don’t turn all of our students into accomplished abstract thinkers by 
the time they leave Allegheny, our students do become more sophisticated thinkers during their four years with us. 

Intellectual Flexibility 

Intellectual flexibility was measured by asking the students to write two essays, one supporting and one opposing 
material provided in a short newspaper article. Scores on each essay can range from -4 to +6. In this particular 
measure we can not only observe changes over time as we did with the ATFR but we have some national normative 
data for comparison. 

Table 1 presents the results of the Analysis of Argument data. There are several interesting as well as several curious 
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findings indicated in this table. First note the Attack scores. 

The Attack scores not only improved from the freshmen to the sophomore year (-.87 to -.76) but these scores are 
below (that is better than) all categories of the norm group. (The “Ivy Colleges’’ norm group included students from 
several schools but most were from Harvard University. The mean SAT score for this group (recentered) was 1,420. 
Mean SAT (recentered) for the “State Teachers Colleges” sample was 1,040 while that for the “Community 
Colleges” was 1,020.) 

On the other hand, the scores for the Defense essay are consistently poorer than those in the norm group and got 
worse as students move from their freshmen to senior years! 

However, if we keep in mind the nature of the essays being written we may understand this outcome. The Attack 
essays were written to attack the point being made in the article that students were too immature to drink and that 
colleges must intervene to reduce the negative outcomes that follow from drinking by college age students. The 
Defense essay would need to support that argument. 

In may be that this particular topic was simply too emotional and too close to the hearts of many of our students and 
they therefore simply could not formulate reasonable and cogent arguments to support a position that was so far 
from their own views. 

Adult Cognitive Development 

Table 2 presents a summary of the data for the various subscales on the SAID. Several interesting findings are 
evident in this table. First, not only are Allegheny students less absolute in their thinking when they arrive when 
compared to the norm group but they become less absolute in their thinking the longer they stay at Allegheny. 
(Statistical tests on this data are difficult since the number of students who presented scores at all three 
measurements is relatively small. However, there is a significant decrease on the SAID A scale from freshmen to 
senior years, /(84) = -2.32, p < .02). 

The relativism subscale presents another interesting finding. Here we see that our students' scores are virtually 
identical to the norm group and they don't change over time. 

The most positive finding can be seen on the evaluativism subscale. Here our students begin well below the norm 
group but their scores show significant increases over time (freshmen to senior change, /(84) = 5.62,/? < .01). 

Initial Interest 



As part of the admission process, students are asked to indicate the field of study they hope to pursue at Allegheny. 
These initial interests were classified into one of the three divisions of the College plus an Undecided category. As 
can be seen in the following tables, there are some measures where students differ according to their initial interest 
and as well as other measures where there were no differences. 

Differences among groups based on initial interest were done using one-way analysis of variance for independent 
groups with a Tukey HSD follow-up test were appropriate. Results are presented in terms of homogenous subsets. 

ATFR . Significant differences were found between initial interest groups on their ATFR scores (F(3,1276) 
= 6.78,/? < .01). As can be seen in Table 3, students who express an initial interest in the Natural Sciences have 
significantly higher ATFR scores than those with interests in the Social Sciences (Undecided and Humanities 
students fall between the two extremes). This suggests that those students who express an initial interest in the 
natural sciences come to Allegheny with more mature thinking skills than those who initially express an interest in 
the social sciences do. 



Analysis of Argument: Attack . This data indicates that students who express an initial interest in the 
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humanities write significantly better essays than those interested in the natural sciences (F (3,1 174) = 3.89,/? < .01). 
See Table 4 for the details. 

Analysis of Argument: Defense . No significant differences were found in the Defense scores based on the 
initial interest of students (F (3,1204) = 0.47, /?> .70). 

Adult Cognitive Development . No significant differences were found on any of the subscales on the SAID 
based on initial interest (SAID A, F(2,643) = 0.22,/? > .80; SAID E, F(2,643) = 1.94,/? > .14; SAID R, F(2,643) = 
2.69,/? >.68). 

SAT scores . An interesting, but not unexpected, pattern can be seen in the distribution of SAT scores for 
groups with different initial interests. Specifically, those interested in the natural sciences have higher SAT math 
scores (Table 5, F(3,1388) = 13.84,/? < .01) while those interested in the humanities had the higher SAT verbal 
scores (Table 6, F(3,1388) = 12.59,/? < .01). When the scores are combined, we see essentially two groups: natural 
science and humanities students with high scores and social science and undecided students with lower scores 
(Table 7, F(3,1573) = 12.88,/? < .01). 

Grades . Although there are no significant differences among the cumulative GPAs for those expressing 
different interests as freshmen (F(3,1584) = 1.65,/? > .18), those who listed their initial interest as Undecided 
achieved a lower grade in LSW200 than those who expressed interest in a specific discipline (F(3,1282) = 3.45,/? < 
. 02 ). 



High school Rank and Size . Size of high school was unrelated to the initial interest of students (F(3,1428) 
= .78,/?> .50) but high school rank did (F(3,1428) = 4.17,/? < .01). Those who have initial interest in the natural 
sciences rank, on average, 42 in their class while those with other interests rank on average at about 50. 

Academic Major 

Now that we have some idea of the differences among students based on their initial interest, we are presented with 
an obvious question: Is this pattern of differences present when student chose their academic major? To answer this 
question, differences among the same set of freshmen scores were examined for the division of major at. In 
addition, since some of the measures were repeated, differences among division of major were also examined for 
those tests given to seniors as well. 

ATFR . As can be seen in Table 8, the same pattern observed when the freshmen ATFR was examined as a 
function of initial interest can be seen when academic major is examined. That is, natural science students had the 
highest scores while those who eventually majored in the humanities or social sciences had significantly lower 
scores (F(2,1046)= 16.05,/? < .01). 

A slightly different pattern is seen when ATFR scores taken from seniors are examined. Here three groups emerge 
(Table 9). Those who majored in the natural sciences still score the highest but now the humanities and social 
science students diverge with those who major in the humanities scoring below those who majored in the social 
sciences (F(2,399) = 12.20,/? < .01). 

Analysis of Argument: Attack . The distribution of freshmen analysis of argument, attack scores by major 
is the same as that seen for initial interest: Humanities majors scored the best, natural science students the worst, and 
social science students fell in the middle (Table 10, F(2,957) = 3.70,/? < .05). However, when analysis of argument 
scores, attack, for seniors are examined we see no difference among the major divisions (F(2,382) = 1.85,/? > .10). 

Analysis of Argument: Defense . No significant differences were found among academic majors for either 
the analysis of argument, defense scores for freshmen (F(2,999) = 0.80,/? > .45) or for seniors (F(2,309) = 1.36,/? > 
.26). 
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Adult Cognitive Development . No significant differences were found on any of the subscales on the SAID 
based on major for any of the SAID scores taken as freshmen (SAID A, F(2,643) = .22, p > .80; SAID E, F(2,643) = 
2.69, /7> .07; SAID R, F(2,643) = 1.94,/7> .14) or as seniors (SAID A, F(2,96) = 1.21,/?> .30; SAID E, F(2,96) = 
.16,/7> ,85; SAID R, F(2,96) = 1.99,/? > .14). 

SAT scores . The pattern observed for initial interest and SAT Math was again seen when major and SAT 
math sores were examined but the differences became even more pronounced. That is, those majoring in the natural 
had significantly higher SAT Math scores than those who majored in either the humanities or the social sciences. 
(Table 11, F(2,l 145) = 34.57,/? < .01). Although the size of the differences is not as pronounced, differences among 
majors on SAT Verbal scores parallel those for initial interest (F(2,l 145) = 4.68,/? < .01) with those in the 
humanities scoring the highest. (See Table 12). Finally, the same pattern seen in initial interest for SAT Total are 
also seen with major (F(2,1271) = 12.60,/? < .01). As seen in Table 13, those majoring in the natural sciences had 
SAT Total scores significantly higher than those in the other divisions. 

Grades . There are significant differences in cumulative GPA among the majors, with those in the 
humanities having the lowest grades (Table 14, F(2,1276) = 8.32, /? < .01). A related finding is that those majoring 
in the natural sciences also had the highest grades in LSW200 (Table 15, F(2,1215) = 6.18,/? < .01). 

High school Rank and Size . As with initial interest, high school size was unrelated to major (F(2,l 167) = 
.69,/?> .50) but high school rank was: Natural science majors ranked significantly higher in their high school 
classes than did those majoring in the social sciences or humanities (Table 16, F(2,l 167) = 17.68) =/? < .01). An 
interesting figure to note is that the average high school class rank for freshmen interested in science is about 42 
while the class rank of those who complete a degree in science is the much better 32. 

Graduation Rates 



Now that we know that students differ on some, but not all, of the measures taken in this study as a function of 
initial interest and academic major, we can begin to explore other relationships. For example, do students who 
graduate differ from those who do not complete their degree on any of these indicators taken during the first week of 
the freshmen year? 

Table 17 provides information to help us answer this question. As can be seen, the only consistently reliable 
measures that differentiates between those who graduate and those who did not is the ATFR. (There are statistically 
reliable differences reported on the SAID E, the Evaluativist scale. However, since this test provided no other 
significant relationships among any of the variables studied, it is likely that this result is a statistical anomaly.) 

Table 18 includes similar statistics for graduates and non-graduates based on their SAT scores. Those who 
completed their education at Allegheny outscored those who did not in all categories. 

Table 19 presents differences between those who graduated and those who did not on several additional variables, 
including high school rank and cumulative GPA. Significant differences are evident for all of the variables except 
size of high school. One note of caution, cumulative GPA for those who graduated and those who did not graduate 
are based on different numbers of courses, with those who graduated including many more advanced courses in a 
major where GPAs are often higher than those in the first two years general education courses. 

One interesting finding concerning graduation is that the graduation rate for women is significantly higher than that 
for men (.Y^(l) = 21.89,/? < .01). About 75% of the women in this sample completed their degree while only 69% of 
the men did. 

It is also interesting to note that graduation rates did not differ among students based on their initial interest (X\3) = 
.721,/? > .89). That is, about 1/3 of the matriculating students did not graduate, no matter what field of inquiry the 
student expressed interest in upon admission to Allegheny. There is a small, but marginally significant, trend in 
graduation rates based on major, however: The highest graduation rate is found among social science students (88%) 
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and the lowest among humanities majors (83%) (X\2) = 4.86,/? < .09). (Note that these graduation rates only apply 
to those who have declared a major so they are higher than the rates based on all incoming students.) 

And finally, although the sample size is small so we should be cautious about the finding, the graduation rate for 
white students was significantly higher than that for non-white students (A"^(5) = 31.30,/? < .01). While the 
graduation rate for whites was about 70%, that for minorities was close for 50%. 

Change in Interests 

But what about those students who change their mind about a major? That is, are students who come to Allegheny 
with an interest in the natural sciences but end up majoring in the humanities different from those who stick with 
their original choice? The answer to that question is somewhat complex, in that in some ways students with 
changing interests are the same as those who do not change while on some other measures there are interesting 
differences. 

There are two ways we can consider change in interest. One way is to simply classify students into one of two 
groups, those who changed divisions from initial interest to major and those who did not change. Second, we could 
examine those changes in more detail. For example, are there differences on any of our measures between those 
who come with an interest in the natural sciences but end up majoring in the humanities and those who come with an 
interest in the humanities and continue that interest through a major? 

Changed division of interest or not . Let's begin by comparing those who continued to a major in the 
division of their initial interest ("did not change") with those who majored in a division different from their 
expressed initial interest ("changed"). Table 20 presents a summary of the variables where there were statistically 
reliable different between the two. Perhaps not surprisingly, in every case, those who ended up in a major different 
from their initial interest scored lower on the variable in question (SAT math, SAT verbal, SAT total, high school 
rank, grade in LSW200, cumulative GPA, and score on the Arlin Test of Formal Reasoning). 

On the other hand, reliable differences were not obtained for the analysis of argument measures (Attack or Defense) 
or any of the subscales on the SAID. 

Many more students who remained with their initial interest as a major graduated when compared to those who 
changed their initial interest (A"^(l) = 262.3,/? < .01). The percentages are interesting here. More than 85% of those 
who majored in the same division as their initial interest graduated while only about 50% of those who changed 
divisions finished with a degree. 

Males are no more likely than females to change their division of interest (A"^(l) = 1 .23, p > .27) and there is no 
difference in division change among the ethnic groups included in this sample (A"^(5) = 5.09,/? > .40). 

Initial interest/division of major . Now let's examine the in more detail the changes that students made as 
their interests shifted from initial interest to major. Initial interests were classified into one of four categories 
(natural science, social science, humanities, and undecided). Since there are three possible divisions for majors, 
there are 12 possible combinations ranging from humanities/humanities to natural science/humanities to 
undecided/humanities. 

Note: In the discussion below H/NS means that a student began with an interest in the humanities but majored in the 
natural sciences, UD/SS is a student who was undecided upon admission but majored in the social sciences, etc. 
Major, by the way, does not necessarily mean that the student graduated; either the student graduated with the stated 
major or he or she had declared that major before he or she left the College. 

ATFR . There is a significant but complex relationship between freshmen ATFR scores and change 
of division of major (F(1 1,1019) = 4.34,/? < .01). In general it appears that those who begin with an interest in the 
social sciences or who end up majoring in the social sciences had the lowest freshmen Arlin scores (mean ATFR 
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scores for SS/H = 16.19, SS/SS = 17.88, and UD/SS = 18.07) while those who end up the natural sciences have the 
highest scores (mean ATFR scores for H/NS = 20.14, NS/NS = 20.20, and SS/NS = 21.65). The similar pattern held 
for the ATFR given to seniors (F(l 1,381) = 4.22,/? < .01) but here those with majors in the humanities and the 
social sciences scored below those majoring the natural sciences (mean ATFR SS/H = 14.17, H/H = 17.73, and 
H/SS =19.13; UD/NS = 21.65, NS/NS = 22.39, and SS/NS = 23.40). 

Analysis of argument: Attack . Although there significant differences among the initial 
interest/major groups on the Attack portion ofthe analysis of argument test (F(l 1,932) = 2.54,/? < .01), but there is 
no discernable pattern in the results. The lowest scores were obtained by SS/NS (mean = -1.14), NS/SS (mean - 
1.12), and SS/H (mean = -1.12) while the highest scores were obtained by NS/H (mean = -.64), H/H (mean -.56), 
and H/SS (mean = -.26). No differences were found for the analysis of argument: Attack essays written by seniors 
(F(l 1,367)= 1.18,/? >.30). 

Analysis of argument: Defense . There were no significant differences for Defense essays written 
by freshmen (F(l 1,973) = .19,/? > .99) or by seniors (F(l 1,294) = .78,/? > .66). 

Adult Cognitive Development . No differences were evidenced on any of the SAID subscales for 
either freshmen or seniors (SAID A, freshmen, F(1 1,621) = 1.67,/? > .07; SAID A, seniors, F( 10,86) = 1.51,/? > .15; 
SAID E, freshmen, F(1 1,621) = .92,/? > .52; SAID E, seniors, F(10,86) = .39,/? > ..94; SAID R, freshmen, 

F{\ 1,621) = .79,/? > ..65; SAID R, seniors, F(10,86) = 1.37,/? > ..21). 

SAT: Math . Perhaps not surprisingly, those who majored in the natural sciences, no matter what 
their initial interests, had significantly higher SAT Math scores than those who majored in the humanities (Table 21, 
F(ll,1116) = 7.58,/?<.01). 

SAT: Verbal . Although the picture is not quite as clear as with the Math scores, those with initial 
interests in or who majored in the humanities had higher SAT Verbal scores than those with other interests or majors 
(F(l 1,1 1 16) = 4.15,/? < .01). The lowest scores were obtained by UD/H (mean = 631), NS/SS (mean = 635), and 
UD/SS (mean = 638) while the highest SAT Verbal scores were obtained by NS/H (mean = 669), H/SS (mean = 
672), and H/H (mean = 675). 

SAT: Total . The addition of Math and Verbal scores results in significant but not easily 
understood differences (F(l 1,1241) = 4.97,/? < .01; see Table 22). The differences is scores is striking (1150 to 
1266) but no one division or no one pattern of changing interests would appear to be evident although it is 
interesting to note that some of the highest scores in the table are for students with initial interests in the humanities 
who ended up majoring in the social or natural sciences. 

Grades . Since we know that those who major in the natural sciences have higher GPAs than those 
in the humanities, is it not surprising to see that finding replicated here (Table 23, F(1 1,1246) = 3.49,/? < .01). 
However, note the finding that those with the highest mean GPA are those with an initial interest in the humanities 
who graduated with a major in the social sciences. 

LSW2Q0 . A clearer pattern emerges when we look at the grades students obtained in LSW200. 
Those who majored in the natural sciences had significantly higher grades than those with majors in the other 
divisions (F(l 1,1 186) = 2.67,/? < .01, Table 24). 

High school Rank and Size . No reliable differences were found among the initial interest/major as 
a function of high school rank (F(l 79,969) = 1.14,/? > .1 1) nor size of high school (F(395,753) = .95,/? > .71). 

Gender . There is a significant relationship between gender and initial interest/major {X\\ 1) = 

1 9.74, /? < .05). For most of the groupings (SS/SS, NS/H, etc.) approximately 53% of those in the group are female 
and 47% male (the over all sample is divided 52.4% female and 47.6% male). However, there is one group that had 
significantly more females (H/H, 68% female) and another with significantly more males (UD/SS, 55% male). 
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Ethnic Group . Although there is a significant relationship between initial interest/major and 
ethnic group (X\55) = 76.64,/? ,.05), there are only two cells that differed much from the overall trend (92% white 
and 8% no white). In both cases (H/SS and SS/H) significantly more non-white students are found in these 
groupings (82% white and 22% non-white). 

Graduation Rate . There is no significant relationship between initial interest/major and graduation 
rate (X\l 1) = 8.73,/? < .64). That is, no group (H/SS, UD/SS, etc.) was more or less likely to graduate than any 
other group. 



Initial interest/major . An interesting and statistically significant relationship exists between initial 
interest and major (X\6) = 432.5, p < .01). As can be seen in Table 25, those who express an initial interest in the 
natural sciences are most likely to change, although those with an initial interest in the humanities are close behind. 
On the other hand, those who express an initial interest in the social sciences are least likely to major in another 
division. Finally, almost half of those who indicate that they are undecided upon admission to Allegheny ended up 
majoring in the social sciences. 

Gender 



Although gender has been considered in several earlier analyses, in this section we will examine differences, if any, 
between men and women in our sample. 

Table 26 provides information on gender differences on some basic demographic variables. Men out perform 
women on SAT Math while women outscore men on SAT Verbal and high school class rank. No gender differences 
were evidenced in SAT Total or high school size. 

Differences on measures specific to this study are shown in Table 27 and Table 28. In Table 27 we see significant 
differences at all three measurement times on the ATFR. In all cases, males outscore females. Freshmen males 
also do significantly better than freshmen females on the Analysis of Argument: Attack essay and as sophomores on 
the Defense essay. 

Table 28 presents some interesting results for the various subscales of the SAID. At all three measurement times, 
women were significantly less absolute in their thinking than men and they were more relativistic (differences for 
seniors, SAID R, was not statistically significant, however). However, as freshmen and sophomores, the men had 
higher scores on evaluativism. 
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Figure 1. Mean number of students in each stage of Formal Operations 
for freshmen, sophomores, and seniors. 




Figure 2. Mean number of freshmen, sophomores, and seniors 
at each level of cognitive functioning. 
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Table 1. Mean Analysis of Argument scores for freshmen, sophomores, and seniors 

for attack and defense essays. 



Reference Group 




Attack 


Defense 


Total 






Freshmen 




-.87 


-1.68 


-2.55 






Sophomores 




-.76 


-1.77 


-2.54 






Seniors 




-.76 


-1.86 


-2.62 






Norm Group 














Ivy Colleges 




-.95 


-1.30 


-2.25 






State Teachers Colleges 




-1.50 


-1.10 


-2.60 






Community Colleges 




-2.25 


-1.15 


-3.40 






Note: Normative data from Winter, McClelland, and Stewart (1981), page 65. 








Table 2. Mean scores for the Scale of Adult Intellectual Development (SAID) 








Reference Group 


Absolutism 


Relativism 




Evaluativism 




Mean 


SD 


Mean SD 




Mean 


SD 


Freshmen 


59.9 


16.7 


94.4 17.6 




93.4 


17.4 


Sophomores 


57.9 


16.6 


92.8 15.4 




94.8 


13.4 


Seniors 


55.6 


15.4 


93.7 11.7 




98.6 


10.7 


Norm Group 


62.4 


16.7 


93.6 15.3 




99.5 


13.2 


Norm group of 446 university students. 












Table 3. Homogeneous subsets for initial interest by division on the Arlin Test of Formal Reasoning. 








Subset for alpha = 


.05 






Initial Interest by Division 




N 


1 


2 






Social Science 




262 


17.87 








Humanities 




162 


18.32 


18.32 






Undecided 




214 


18.38 


18.38 






Natural Science 




642 




19.47 








15 



16 



Table 4. Homogeneous subsets for initial interest by division on Analysis of Argument: Attack. 



Initial Interest by Division 


N 


Subset for alpha = .05 
I 2 


Natural Sciences 


564 


-.97 


Social Sciences 


236 


-.89 -.89 


Undecided 


199 


-.80 -.80 


Humanities 


159 


-.60 



Table 5. Homogeneous subsets for initial interest by division on SAT Math. 






Subset for alpha = .05 


Initial Interest by Division 


N 


I 2 


Social Sciences 


273 


570 


Undecided 


231 


578 


Humanities 


187 


578 


Natural Science 


701 


594 


Table 6. Homogeneous subsets for initial interest by division on SAT Verbal. 






Subset for alpha = .05 


Initial Interest by Division 


N 


I 2 


Undecided 


231 


635 


Social Sciences 


273 


639 


Natural Science 


701 


649 


Humanities 


187 


671 




16 



17 



Table 7. Homogeneous subsets for initial interest by division on SAT Total. 







Subset for alpha = 


.05 


Initial Interest by Division 


N 


1 


2 


Social Science 


327 


1174 




Undecided 


260 


1194 




Natural Science 


783 




1221 


Humanities 


207 




1224 


Table 8. Homogeneous subsets 


for major by division on the freshmen ATFR. 








Subset for alpha = 


.05 


Major by Division 


N 


1 


2 


Humanities 


244 


18.15 




Social Science 


425 


18.39 




Natural Science 


380 




20.15 



Table 9. Homogeneous subsets for major by division on the senior ATFR. 



Subset for alpha = .05 



Major by Division 


N 


1 


2 3 


Humanities 


71 


18.61 




Social Science 


178 




20.47 


Natural Science 


153 




22.27 


Table 10. Homogeneous subsets 


for major by division 


on the freshmen 


analysis of argument: attack 






Subset for alpha = .05 


Major by Division 


N 


1 


2 


Natural Science 


340 


-.99 




Social Science 


384 


-.85 


-.85 


Humanities 


236 




-.70 




17 



18 



Table 1 1. Homogeneous subsets for major by division for SAT Math 



Subset for alpha = .05 



Major by Division 



N 



2 



Humanities 


266 


572 




Social Science 


460 


578 




Natural Science 


422 




603 


Table 12. Homogeneous subsets for major by division for SAT Verbal. 






Subset for alpha = 


.05 


Major by Division 


N 


1 


2 


Social Science 


460 


644 




Natural Science 


422 


651 


651 


Humanities 


266 




659 



Table 1 3. Homogeneous subsets for major by division for SAT Total. 







Subset for alpha = .05 


Major by Division 


N 


1 2 


Humanities 


307 


1199 


Social Science 


508 


1202 


Natural Science 


459 


1237 


Table 14. Homogeneous subsets for major by division cumulative grade point average. 






Subset for alpha = .05 


Major by Division 


N 


1 2 


Humanities 


307 


2.89 


Social Science 


510 


3.00 


Natural Sciences 


462 


3.05 



ERIC 



18 



19 



Table 15. Homogeneous subsets for major by division grade in LSW 200. 







Subset for alpha = .05 


Major by Division 


N 


1 2 



Social Science 


491 


3.14 




Humanities 


283 


3.16 




Natural Sciences 


444 




3 



Table 16. Homogeneous subsets for major by division for high school class rank.. 







Subset for alpha = .05 


Major by Division 


N 


1 2 


Natural Science 


423 


31.78 



Social Science 472 47.10 

Humanities 275 55.93 



Table 17. Mean scores on basic measures of intellectual functioning for students who graduated and those 
who did not graduate. 



Measure 


Mean 




t 


Significance Level 




Graduated 


Did Not Graduate 




(2-tailed) 


Arlin 


19.06 


18.32 


2.49 


.013* 


Attack 


-.83 


-.95 


1.63 


.103 


Defense 


-1.64 


-1.77 


.82 


.415 


SAIDA 


59.36 


60.89 


-1.25 


.212 


SAIDE 


94.54 


91.17 


2.64 


.008* 


SAIDR 


94.88 


93.41 


1.13 


.251 


* differences significant, /? < .05. 
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Table 18. Mean SAT scores for students who graduated and those who did not graduate. 



Measure 


Mean 

Graduated 


Did Not Graduate 


t 


Significance Level 
(2-tailed) 


SAT Math 


587 


578 


2.76 


.006* 


SAT Verbal 


653 


635 


5.26 


.000* 


SAT Total 


1219 


1183 


4.75 


.000* 



* differences significant, p < .05 

Table 19. Mean scores on demographic variables for students who graduated and those who did not 
graduate. 



Measure 


Mean 

Graduated 


Did Not Graduate 


t 


Significance Level 
(2-tailed) 


HS rank 


40.45 


61.27 


-6.44 


.000* 


HS class size 


227.96 


234.29 


-.80 


.422 


LSW200 grade 


3.26 


2.60 


14.21 


.000* 


Cum GPA 


3.08 


2.33 


25.48 


.000* 



* differences significant,/? < .05 

Table 20. Mean scores on several variables for students who majored in a division different from that 
expressed as an initial interest and those who did not major in the division of their initial interest. 



Measure 


Mean 
Did not change 


Changed 


t 


Significance Level 
(2-tailed) 


SAT math 


591 


579 


4.09 


.000* 


SAT verbal 


652 


645 


2.04 


.041* 


SAT total 


1222 


1198 


3.66 


.000* 


HS rank 


39.6 


51.5 


-3.94 


.000* 


LSW200 grade 


3.24 


3.04 


5.19 


.000* 


Cum GPA 


3.04 


2.72 


9.94 


.000* 


ATFR (freshmen) 19.23 


18.54 


2.41 


.016* 



* differences significant, p < .05 
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Table 21. Homogeneous subsets for initial interest /major on SAT: Math. 



Initial Interest/Major 


N 


Subset for alpha = 
1 


.05 

2 


SS/H 


43 


551 




UD/H 


52 


566 


566 


H/H 


90 


573 


573 


ss/ss 


166 


575 


575 


UD/SS 


88 


577 


577 


NS/SS 


147 


580 


580 


H/SS 


47 


586 


586 


NS/H 


76 


588 


588 


UD/NS 


43 




600 


SS/NS 


16 




601 


H/NS 


14 




601 


NS/NS 


346 




604 



Table 22. Homogeneous subsets for initial interest /major on SAT: Total. 






Subset for alpha = 


.05 


Initial Interest/Major 


N 


1 


2 


SS/H 


52 


1149 




SS/NS 


20 


1190 


1190 


UD/H 


57 


1191 


1191 


SS/SS 


186 


1193 


1193 


UD/SS 


97 


1196 


1196 


NS/SS 


164 


1197 


1197 


H/H 


103 


1211 


1211 


NS/H 


90 


1220 


1220 


UD/NS 


46 




1230 


NS/NS 


375 




1239 


H/SS 


48 




1260 


H/NS 


15 




1266 




21 



22 



Table 23. Homogeneous subsets for initial interest /major for GPA. 



Initial Interest/Major 


N 


Subset for alpha = 
1 


.05 

2 


SS/H 


52 


2.75 




UD/H 


57 


2.80 




NS/H 


90 


2.88 


2.88 


NS/SS 


164 


2.92 


2.92 


UD/SS 


98 


2.94 


2.94 


H/NS 


15 


2.95 


2.95 


H/H 


103 


3.01 


2.01 


ss/ss 


187 


3.03 


3.03 


UD/NS 


47 


3.05 


3.05 


NS/NS 


377 


3.05 


3.05 


SS/NS 


20 


3.07 


3.07 


H/SS 


48 




3.14 



Table 24. Homogeneous subsets for initial interest /major for grades in LSW200.. 






Subset for alpha = 


.05 


Initial Interest/Major 


N 


1 


2 


UD/H 


56 


2.97 




UD/SS 


96 


3.06 


3.06 


NS/SS 


156 


3.06 


3.06 


SS/H 


49 


3.14 


3.14 


SS/SS 


178 


3.18 


3.18 


UD/NS 


45 


3.18 


3.18 


NS/H 


83 


3.19 


3.19 


H/H 


90 


3.24 


3.24 


N/SS 


46 


3.28 


3.28 


NS/NS 


362 


3.28 


3.28 


H/NS 


14 


3.31 


3.31 


SS/NS 


20 




3.40 




22 



23 



Table 25. Initial interest by division as a function of major division. 



Division of Initial Interest 




Division of Major 




Humanities 


Natural Sciences 


Social Sciences 


Humanities 


103 


15 


48 




(62%) 


(9%) 


(29%) 


Natural Science 


90 


377 


164 




(14%) 


(60%) 


(26%) 


Social Science 


52 


20 


187 




(20%) 


(8%) 


(72%) 


Undecided 


57 


47 


98 




(28%) 


(23%) 


(49%) 



Table 26. Mean scores for men and women on SATs, highschool rank and highschool size. 



Measure 


Mean 

Women 


Men 


t 


Significance Level 
(2-tailed) 


SAT Math 


575 


594 


-6.33 


.000* 


SAT Verbal 


656 


640 


4.66 


.000* 


SAT Total 


1210 


1205 


.78 


.435 


Highschool Rank 


38 


55 


-5.65 


.000* 


Highschool Size 


227 


233 


-.80 


.423 



* differences significant,/? < .05 
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Table 27. Mean scores for men and women on ATFR and analysis of argument scores. 



Measure Mean 

Women 


Men 


t 


Significance Level 
(2-tailed) 


ATFR, freshmen 


18.2 


19.6 


-4.82 


.000* 


ATFR, sophomores 


18.9 


19.7 


-2.22 


.026* 


ATFR, seniors 


20.20 


21.84 


-2.98 


.003* 


Attack, freshmen 


-.95 


-.78 


-2.41 


.016* 


Attack, sophomores 


-.78 


-.74 


-.46 


.648 


Attack, seniors 


-.75 


-.74 


-.10 


.920 


Defense, freshmen 


-1.66 


-1.69 


.16 


.870 


Defense, sophomores 


-1.84 


-1.70 


-2.40 


.017* 


Defense, seniors 


-1.88 


-1.83 


-.80 


.421 



* differences significant,/? < .05 

Table 28. Mean scores for men and women on analysis of argument 


scores. 




Measure 


Mean 

Women 


Men 


t 


Significance Level 
(2-tailed) 


SAID A, freshmen 


59.29 


64.60 


-7.30 


.000* 


SAID A, sophomores 


52.85 


65.05 


-8.40 


.000* 


SAID A, seniors 


51.48 


63.18 


-3.99 


.000* 


SAID E, freshmen 


95.03 


92.54 


2.08 


.038* 


SAID E, spohomores 


96.20 


93.60 


2.19 


.035* 


SAID E, seniors 


98.31 


98.84 


-.23 


.815 


SAID R, freshmen 


96.91 


91.88 


4.15 


.000* 


SAID R, spohomores 


95.42 


90.78 


3.44 


.001* 


SAID R, seniors 


93.79 


92.64 


. 53 


.596 



* differences significant,/? < .05 
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